Lines of Latitude & Longitude
This week’s assignment will encompass the following concepts covered in Class 2 lecture demo & lab:
A mapped GCS coordinate
Latitude as rungs on ladder
Longitude as telephone poles
This week’s reading will be primarily in the Essentials of Geographic Information Systems textbook, with 1 additional reading accessible as an online PDF.
The Class 2 quiz (09/20/2021) will features 10 questions covering content in the textbook.
Readings:
In the following assignment steps the NYC’s 311 Service Requests for Covid Social Distancing is utilized as an example dataset. Generally the same steps apply to all 311 themes available. For this assignment, choose a theme and a time period filter for your own assignment submission.
In order to choose a theme, you must obviously know which themes are available. In the 311 dataset, themes are organized by Complaint Type first; with subcategories that are listed as Descriptor. A third column is provided for the theme preview titled Unique Key, which in this case is a count of records per Complaint Type + Descriptor. To no one’s surprise, Noise - Residential + Loud Music/Party has the largest count at 1,665,443 records dating back to 2010.
To download the theme preview, navigate to, and download:
In this assignment, we will map the 311 data, conduct a spatial operation then visualize the results both in cartographic and chart outputs. Often GIS output will feature a combination of aspects, attributes or dimensions of a dataset. A recent example of this approach to data visualization from National Geographic - Monumental Undertaking shows first the temporal dimension of Confederate statues, followed by their spatial distribution:
Example of data presentation - chart + map
The following mapping assignment utilizes the spatial distribution of Social Distancing complaints as an assignment example, alongside an aggregate count per NYC borough. By visualizing both the disaggreate data points and aggregate count, the final product allows a meaningful count in addition to spatial distribution alone.
To Start, first acquire the data in .csv format, then map it against the 5 boroughs in NYC. The Latitude and Longitude spatial attributes in the dataset will be used to transform the tabular dataset into a spatial dataset with geometry - .csv to .shp. Along the way, a total complaint count per borough will be determined, and finally a rate of complaint per borough. A map product is then developed that shows the pattern of the data and charts the relative complaint count per borough.
To start, the following filter query and rate formula will be utilized in the assignment:
Social Distancing.shape_area * 0.00000003587006428 = square mileNote: for filtering, a combination of
Complaint TypeandDescriptorcan be utilized to narrow results.
NYC OpenData
311 Service Requests
Dataset, not Data Lens instance:311 Dataset
View Data button selection:View Data
Descriptor is Social Distancing:Applying Data Filter
Social Distancing Descriptor attribute:Record Results
.csv to a folder directory for Part II. Accept the default naming convention of the .csv:Download Record Results
.csv in Excel and navigate to right-end of the file. Here you will see Latitude and Longitude. These two columns will be utilized to read in the locational data for the creating of geometry (points) within QGIS:Locational Data
.csv of the 311 data as of 01/29/21. Only use this .csv as testing data, not data for the assignment per se:Class 2 Assignment Data
.csv service requests utilizing the Delimited Text data source option. Make sure to populate the tool as seen below and rename the layer simply as 311_SR:Delimited Text - QGIS
0.25 size in black, and the boroughs are a light neutral color of your choice:NYC Borough Geography
Note: There will be some stray points outside NYC area. These are incorrect lat/lon data points wherein the lat/lon was recorded incorrectly in the 311 dataset. Disregard these points. Also now is the time to begin to save your
.qgsfile on a regular basis in order to avoid duplicate work in the event of a software crash.
wgs84 coordinate system. We can change that easily so that the ‘shape’ is much more accurate. To do so, we will utilize the map projection designed for the NYC area (State Plane system). First click on the CRS button, lower right of Map Canvas. Once done, search for 3104 which is a short code for the correct map projection for NYC. Then click Add at bottom of tool window. The shape of the data will automatically update. This is a type of ‘on the fly’ projection; we are changing the Map Canvas coordinates, but we are leaving the actual .prj file on each of the data sources unchanged. This is a ‘quick and dirty’ yet efficient cartographic kludge (hack):Location of CRS reference - lower right of Map Canvas
CRS Filter
Note: you may run into a CRS projection issue with the polygon feature for NYC boroughs. IF your features disappear from your Map Canvas as you perform an ‘on the fly’ projection for the data frame, the following video will solve this problem:
Next, we want know how many complaints exist per borough expressed in the following structured question:
Note: for this second question, perhaps a more insightful query would be rate of public complaints per 10K population. Certainly we can answer this question also, but we would need population data which we will cover in depth in Class 4 - Thematic Mapping.
Utilize the Count points in polygon overlay operation to get a total count of complaint records per borough. This tool is available from the Main Menu:
Vector > Analysis Tools > Count Points in Polygon
Count Points in Polygon Operation
Counts in Polygon Tool Parameters
COUNT. Immediately export this new layer as a .shp into your working directly, entitled 311_borough:Export Operation
Export Operation as .shp
Note: see that the new
.shpnow contains a field column entitledNUMPOINTSin integer format (int8) - this is the count per borough.
311_borough.shp. To do this navigate to the Toolbar and choose the Field Calculator tool:QGIS Field Calculator
Make sure 311_borough is active in the layers panel. Create a new field sq.mile and populate with the following formula, using Fields and Values > shape_area dropdown:
shape_area * 0.00000003587006428Field Calculator Operation
Note: This formula takes the current areal quanity in sq. feet and converts it to a sq. mile - a unit that is more appropriate to the size of the borough polygons.
Toggle Editing
With the number of complaints per borough (NUMPOINTS) and the sq. miles of each borough (sq.mile), a rate normalized to sq. area can be achieved. Again, open the Field Calculator and use the same process as above to Create a new field, but this time name the field rate.sq.mi and use the following formula:
"NUMPOINTS" / "sq.mile"
Again, save the attribute addition; Toggle Editing to the off position and Save.
Open the attribute table and view the results. Its clear the rate of complaints per sq. mile in Manhattan is highest, even as its total count is lower than Brooklyn. The influence of the outer neighborhoods in Brooklyn is likely bringing down the rate in Brooklyn as a whole. In other words, the most populated areas of Brooklyn are likely on par with the rate of complaints per sq. mile as Manhattan.
Attribute Table with Added Fields
Note: if you are having problems with the Field Calculator at this stage of the assignment, the following video will walk through each step and may be helpful to get back on track:
With the analysis steps complete, we move along to cartographic design and output. We can augment the map design with a graphing feature to good effect. The following components will be featured in this final stage:
To start, make sure the .qgs files is saved.
Note: as there will be 60K points on this map, consider ramping down opacity and keep small symbol size. For the example map, opacity of points has been set at approx. 75%, and the size 0.25.
QGIS Plugins
DataPlotly
DataPlotly toolset
Parameters and Properties
Create Plot in the lower section of the tool window. This creates a preview of the graph:Gears Icon at DataPlotly
HTML expression of the chart. This code can be highlighted, copied and then placed into a text editor (do not use MS Word. In the Class 2 lab, text editors will be discussed. There are many free, accessible editors).HTML DataPlotly Properties
With the code complete and copied safely to a text editor (not MS Word), Create a new layout instance, and Save Project from within Map Layout. Include the following elements:
The first three elements were covered generally in Class 1, Assignment 1 so you should have some familiarity with their creation and design. The fourth element - the Data Plotly graph - is a bit more challenging, but with some practice both valuable and efficient.
To start the graph, click the Add a New HTML to the Layout icon at left and draw out a element box on the Layout Canvas:
Add copied HTML to Layout
HTML code from the text editor into the HTML source section with the Source button activated. Click Return and the chart will appear on the Layout Canvas:New HTML element in Layout Canvas
Actions - Horizontal Properties Positioning
The font style is difficult to change within Data Plotly. Given this constraint, you can either use the same font as that of Data Plotly - Open Sans - or you can find a complimentary font style for your title, legend items, ect.
Download Open Sans font - free - Open Sans Font
Review the Assignment Example before finalizing your assignment. You are encouraged to experiment and choose a different approach than the example, but do indeed include all elements found in the example. Upload your PDF export to CANVAS assignment 2 submission location.
Additional Class 2 Spatial realated data/readings/articles:
A concise definition of bounding box often used in conjunction with lat/lon.
Strava’s fitness tracker heat map reveals the location of military bases
Often lat/lon pairings are expressed in DMS (degree, minute, second) as opposed to Decimal format which is necessary & standard for digital mapping. Find out more on the expression of DMS vs. Decimal format.
To follow, a short tutorial on Degree|Minute|Second conversion to Decimal lat/lon expression:
Further, you can make the DMS>Decimal Degrees and vice versa on a point-by-point basis using an online converter like the following: